The average JDub that doesn't know you or your history is no big deal. To them you’re just another householder who probably isn't interested. However JDubs given your address by your family is another matter entirely. You just know that there will be a report back home as to how you and your wife responded to their visit. That's definitely not a very comfortable position to be in. I certainly am glad that the days of hiding my thoughts regarding the WBTS from my JDub mother are behind me.
Good luck.